Adding support of Azure OPENAI both text embedding and RAG #290
Conversation
…er configuration guidance for Azure OpenAI
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @ShawnXxy, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly expands DeepWiki's AI provider capabilities by integrating full support for Azure OpenAI. It allows the application to seamlessly utilize Azure OpenAI for both text generation and embedding tasks, with the system intelligently detecting and prioritizing Azure if configured. The changes encompass core AI client implementations, robust error handling, dynamic configuration, and comprehensive documentation updates, providing users with greater flexibility and enterprise-grade AI options.
Highlights
- Azure OpenAI Integration: Implemented comprehensive support for Azure OpenAI, enabling its use for both text generation (RAG) and text embedding. The system now intelligently detects and prioritizes Azure OpenAI if configured via environment variables.
- Intelligent API Retry Logic: Introduced custom retry decorators for Azure OpenAI API calls in
api/azureai_client.pythat intelligently parse rate limit errors and wait for the exactretry-afterduration, improving resilience and API interaction efficiency. - Optimized Azure Embeddings: Added dedicated
AzureBatchEmbedderandAzureToEmbeddingscomponents to handle batch embedding requests specifically for Azure OpenAI, including rate limiting considerations and local caching of embeddings for performance. - Dynamic Configuration & Auto-Detection: Updated the configuration system (
api/config.py) to dynamically load Azure-specific settings and automatically set Azure as the default AI provider and embedder if its environment variables are properly configured. - Enhanced User Guidance: Improved
Dockerfilestartup messages andREADME.mddocumentation to clearly guide users on configuring Azure OpenAI, including detailed environment variable explanations and setup steps for both Docker and manual installations. Frontend error messages were also updated to reflect Azure OpenAI as a valid embedding provider option.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request adds comprehensive support for Azure OpenAI, including auto-detection of configuration, specific retry logic for rate limiting, and updated documentation. The changes are well-structured. I've found a few issues in the documentation that need fixing for clarity and correctness, and a bug in the new retry logic that could cause premature failures. Addressing these points will improve the robustness and user experience of the new feature.
| 1. Analyze the2. **Add to your `.env` file** (see `.env.example` for full template): | ||
| ```bash | ||
| AZURE_OPENAI_EMBEDDING_API_KEY=your_api_key | ||
| AZURE_OPENAI_EMBEDDING_ENDPOINT=https://your-resource.openai.azure.come structure |
There was a problem hiding this comment.
There appears to be a copy-paste error in the introduction. The text 2. **Add to your... has been inserted in the middle of the first list item, which makes the sentence incoherent. Please correct this formatting.
| except RateLimitError as e: | ||
| retry_count += 1 | ||
| error_message = str(e) | ||
|
|
||
| # Try to parse the required delay from error message | ||
| retry_delay = parse_azure_rate_limit_error(error_message) | ||
|
|
||
| if retry_delay is not None and retry_count < max_retries: | ||
| log.warning(f"Azure OpenAI rate limit hit. " | ||
| f"Waiting {retry_delay} seconds before retry " | ||
| f"({retry_count}/{max_retries})") | ||
| time.sleep(retry_delay) | ||
| continue | ||
| else: | ||
| # If we can't parse delay or max retries reached, re-raise | ||
| if retry_count >= max_retries: | ||
| log.error(f"Max retries ({max_retries}) reached " | ||
| f"for rate limit error") | ||
| else: | ||
| log.warning("Could not parse retry delay from " | ||
| "error message") | ||
| raise | ||
| except (APITimeoutError, InternalServerError, | ||
| UnprocessableEntityError, BadRequestError) as e: | ||
| # For other errors, use simple exponential backoff | ||
| if retry_count < max_retries - 1: | ||
| retry_count += 1 | ||
| delay = 2 ** retry_count | ||
| log.warning(f"API error: {type(e).__name__}. " | ||
| f"Retrying in {delay} seconds " | ||
| f"({retry_count}/{max_retries})") | ||
| time.sleep(delay) | ||
| continue | ||
| else: | ||
| raise |
There was a problem hiding this comment.
The retry logic here has a couple of issues:
- If a
RateLimitErroris caught but the specific delay cannot be parsed from the error message, the exception is immediately re-raised. This prevents a fallback to exponential backoff, making the retry mechanism less robust. - The retry logic is duplicated and slightly different for
RateLimitErrorand other API errors, which can be confusing and hard to maintain. Theif retry_count < max_retries - 1for other errors seems to allow one fewer retry than intended.
I suggest unifying the exception handling to make it more robust and easier to read. This revised logic first checks for a specific delay for RateLimitError and then falls back to exponential backoff for all other retryable errors, including rate limit errors with unknown message formats. A similar change should be applied to the azure_openai_async_retry_with_delay decorator.
except (RateLimitError, APITimeoutError, InternalServerError,
UnprocessableEntityError, BadRequestError) as e:
# For other errors, use simple exponential backoff
retry_count += 1
if retry_count >= max_retries:
log.error(f"Max retries ({max_retries}) reached for API error: {e}")
raise
delay = None
if isinstance(e, RateLimitError):
delay = parse_azure_rate_limit_error(str(e))
if delay is not None:
log.warning(f"Azure OpenAI rate limit hit. Waiting {delay} seconds before retry ({retry_count}/{max_retries})")
else:
# Fallback to exponential backoff
delay = 2 ** retry_count
log.warning(f"API error: {type(e).__name__}. Retrying in {delay} seconds ({retry_count}/{max_retries})")
time.sleep(delay)
continue| # Optional: Add this if you want to use Azure OpenAI models (auto-detected) | ||
| ## Text Embedding model: | ||
| AZURE_OPENAI_EMBEDDING_API_KEY=your_api_key | ||
| AZURE_OPENAI_EMBEDDING_ENDPOINT=https://your-resource.openai.azure.com | ||
| AZURE_OPENAI_EMBEDDING_DEPLOYMENT=text-embedding-3-large | ||
| AZURE_OPENAI_EMBEDDING_VERSION=2024-12-01-preview | ||
| ## Text Generation model: | ||
| AZURE_OPENAI_API_KEY=your_api_key | ||
| AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com | ||
| AZURE_OPENAI_DEPLOYMENT=gpt4o | ||
| AZURE_OPENAI_VERSION=2024-12-01-preview | ||
|
|
||
| # Optional: Separate embedding endpoint for Azure OpenAI (if different from main endpoint) | ||
| AZURE_OPENAI_EMBEDDING_ENDPOINT=your_azure_embedding_endpoint | ||
| AZURE_OPENAI_EMBEDDING_API_KEY=your_azure_embedding_api_key | ||
|
|
There was a problem hiding this comment.
The documentation for Azure OpenAI environment variables in this section is a bit confusing because AZURE_OPENAI_EMBEDDING_ENDPOINT and AZURE_OPENAI_EMBEDDING_API_KEY are listed twice.
To improve clarity, I suggest restructuring this section to clearly explain the different scenarios, for example:
- Using the same Azure resource for both generation and embeddings.
- Using separate Azure resources for each.
Adding support of Azure OPENAI client to allow both text-embedding and text generation so that the default Google/OpenAI model could be optional.